在本文中，我们提出了一个大型详细的3D面部数据集，FACESCAPE和相应的基准，以评估单视图面部3D重建。通过对FACESCAPE数据进行训练，提出了一种新的算法来预测从单个图像输入的精心索引3D面模型。 FACESCAPE DataSet提供18,760个纹理的3D面，从938个科目捕获，每个纹理和每个特定表达式。 3D模型包含孔径级面部几何形状，也被处理为拓扑均匀化。这些精细的3D面部模型可以表示为用于详细几何的粗糙形状和位移图的3D可线模型。利用大规模和高精度的数据集，进一步提出了一种使用深神经网络学习特定于表达式动态细节的新颖算法。学习的关系是从单个图像输入的3D面预测系统的基础。与以前的方法不同，我们的预测3D模型在不同表达式下具有高度详细的几何形状。我们还使用FACESCAPE数据来生成野外和实验室内基准，以评估最近的单视面重建方法。报告并分析了相机姿势和焦距的尺寸，并提供了忠诚和综合评估，并揭示了新的挑战。前所未有的数据集，基准和代码已被释放到公众以进行研究目的。

Medical image segmentation (MIS) is essential for supporting disease diagnosis and treatment effect assessment. Despite considerable advances in artificial intelligence (AI) for MIS, clinicians remain skeptical of its utility, maintaining low confidence in such black box systems, with this problem being exacerbated by low generalization for out-of-distribution (OOD) data. To move towards effective clinical utilization, we propose a foundation model named EvidenceCap, which makes the box transparent in a quantifiable way by uncertainty estimation. EvidenceCap not only makes AI visible in regions of uncertainty and OOD data, but also enhances the reliability, robustness, and computational efficiency of MIS. Uncertainty is modeled explicitly through subjective logic theory to gather strong evidence from features. We show the effectiveness of EvidenceCap in three segmentation datasets and apply it to the clinic. Our work sheds light on clinical safe applications and explainable AI, and can contribute towards trustworthiness in the medical domain.

基于粒子的系统提供了一种灵活而统一的方法，可以模拟具有复杂动力学的物理系统。大多数现有的基于粒子系统的数据驱动的模拟器采用图形神经网络（GNN）作为网络骨架，因为粒子及其相互作用可以由图节点和图形边缘自然表示。但是，虽然基于粒子的系统通常包含数百千个颗粒，但由于粒子相互作用的数量增加，粒子相互作用的显式建模不可避免地会导致显着的计算开销。因此，在本文中，我们提出了一种基于变压器的新型方法，称为具有隐式边缘（TIE）的变压器，以无边缘方式捕获粒子相互作用的丰富语义。领带的核心思想是将涉及涉及配对粒子相互作用的计算分散到每个颗粒更新中。这是通过调整自我发项式模块以类似于GNN中图表的更新公式来实现的。为了提高领带的概括能力，我们进一步修改了可学习的特定材料的抽象粒子，以将全球材料的语义与本地粒子语义分开。我们评估了不同复杂性和材料不同领域的模型。与现有的基于GNN的方法相比，没有铃铛和哨子，TIE可以在所有这些领域中实现卓越的性能和概括。代码和模型可在https://github.com/ftbabi/tie_eccv2022.git上找到。

多视图子空间聚类传统上专注于集成异构特征描述以捕获更高维度信息。一种流行的策略是从不同视图生成常见的子空间，然后应用基于图形的方法来处理群集。但是，这些方法的性能仍然受到两个限制，即多视图融合模式以及融合过程与聚类任务之间的连接。为了解决这些问题，我们通过细粒度图形学习提出了一种新的多视图子空间聚类框架，可以在不同视图之间讲述本地结构之间的一致性，并比以前的重量规则更精细地集成所有视图。与文献中的其他模型不同，引入了点级图正规化和频谱聚类的重新介绍，以执行图形融合并将共享集群结构一起学习在一起。在五个真实数据集上进行了广泛的实验，表明该框架对SOTA算法具有可比性。

经过验证的多模态融合是提高扬声器跟踪的准确性和稳健性的有效方法，尤其是在复杂的情景中。但是，如何结合异构信息并利用多模态信号的互补性仍然是一个具有挑战性的问题。在本文中，我们提出了一种使用音频和视觉方式的扬声器跟踪的新型多模态感知跟踪器（MPT）。具体地，首先构建基于空间全局相干字段（STGCF）的新型声学图以用于异构信号融合，其采用相机模型将音频线索映射到与视觉提示一致的定位空间。然后，引入了多模态感知关注网络以导出测量受噪声干扰的间歇音频和视频流的可靠性和有效性的感知权重。此外，提出了一种独特的跨模式自我监督学习方法，以通过利用不同方式之间的互补性和一致性来模拟音频和视觉观测的置信度。实验结果表明，该拟议的MPT分别在标准和封闭数据集上实现了98.6％和78.3％的跟踪准确性，其在不利条件下展示了其鲁棒性，并且优于目前最先进的方法。

Masked image modeling (MIM) performs strongly in pre-training large vision Transformers (ViTs). However, small models that are critical for real-world applications cannot or only marginally benefit from this pre-training approach. In this paper, we explore distillation techniques to transfer the success of large MIM-based pre-trained models to smaller ones. We systematically study different options in the distillation framework, including distilling targets, losses, input, network regularization, sequential distillation, etc, revealing that: 1) Distilling token relations is more effective than CLS token- and feature-based distillation; 2) An intermediate layer of the teacher network as target perform better than that using the last layer when the depth of the student mismatches that of the teacher; 3) Weak regularization is preferred; etc. With these findings, we achieve significant fine-tuning accuracy improvements over the scratch MIM pre-training on ImageNet-1K classification, using all the ViT-Tiny, ViT-Small, and ViT-base models, with +4.2%/+2.4%/+1.4% gains, respectively. Our TinyMIM model of base size achieves 52.2 mIoU in AE20K semantic segmentation, which is +4.1 higher than the MAE baseline. Our TinyMIM model of tiny size achieves 79.6% top-1 accuracy on ImageNet-1K image classification, which sets a new record for small vision models of the same size and computation budget. This strong performance suggests an alternative way for developing small vision Transformer models, that is, by exploring better training methods rather than introducing inductive biases into architectures as in most previous works. Code is available at https://github.com/OliverRensu/TinyMIM.

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.

Blind image quality assessment (BIQA) remains challenging due to the diversity of distortion and image content variation, which complicate the distortion patterns crossing different scales and aggravate the difficulty of the regression problem for BIQA. However, existing BIQA methods often fail to consider multi-scale distortion patterns and image content, and little research has been done on learning strategies to make the regression model produce better performance. In this paper, we propose a simple yet effective Progressive Multi-Task Image Quality Assessment (PMT-IQA) model, which contains a multi-scale feature extraction module (MS) and a progressive multi-task learning module (PMT), to help the model learn complex distortion patterns and better optimize the regression issue to align with the law of human learning process from easy to hard. To verify the effectiveness of the proposed PMT-IQA model, we conduct experiments on four widely used public datasets, and the experimental results indicate that the performance of PMT-IQA is superior to the comparison approaches, and both MS and PMT modules improve the model's performance.

Automatic music generation with artificial intelligence typically requires a large amount of data which is hard to obtain for many less common genres and musical instruments. To tackle this issue, we present ongoing work and preliminary findings on the possibility for deep models to transfer knowledge from language to music, by finetuning large language models pre-trained on a massive text corpus on only hundreds of MIDI files of drum performances. We show that by doing so, one of the largest, state-of-the-art models (GPT3) is capable of generating reasonable drum grooves, while models that are not pre-trained (Transformer) shows no such ability beyond naive repetition. Evaluating generated music is a challenging task, more so is evaluating drum grooves with little precedence in literature. Hence, we propose a tailored structural evaluation method and analyze drum grooves produced by GPT3 compared to those played by human professionals, exposing the strengths and weaknesses of such generation by language-to-music transfer. Our findings suggest that language-to-music transfer learning with large language models is viable and promising.